Generating replacement sentences for a particular sentiment

ABSTRACT

Certain aspects of the present disclosure provide techniques for generating a replacement sentence with the same or similar meaning but a different sentiment than an input sentence. The method generally includes receiving a request for a replacement sentence and iteratively determining a next word of the replacement sentence word-by-word based on an input sentence. Iteratively determining the next word generally includes evaluating a set of words of the input sentence using a language model configured to output candidate sentences and evaluating the candidate sentences using a sentiment model configured to output sentiment scores for the candidates sentences. Iteratively determining the next word further includes calculating convex combinations for the candidate sentences and selecting an ending word of one of the candidate sentences as the next word of the replacement sentence. The method further includes transmitting the replacement sentence in response to the request for the replacement sentence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application is a continuation of and hereby claims priority under35 U.S.C. § 120 to co-pending U.S. Pat. Application No. 17/495,681,filed on Oct. 6, 2021, which is a continuation of U.S. Pat. ApplicationNo. 16/459,199, filed on Jul. 1, 2019, now U.S. Pat. No. 11,170,175, theentire contents of each of which are incorporated herein by reference.

INTRODUCTION

Aspects of the present disclosure relate to generation of sentencesusing machine learning techniques.

An important aspect of maintaining software applications for computingdevices (or other types of services) is to provide customer support(such as technical help) to users. In addition to material such as helpguides or frequency asked questions (FAQ) documents, support agents andvarious automated tools may be provided to assist users in real time. Inorder to improve user retention of the information and user satisfactionwith the assistance, and possibly the service as a whole, such real timeassistance should be accurate and should also convey a proper sentiment.However, no tools currently exist to automatically rephrase a sentenceto retain meaning but change sentiment to match a specified sentiment,and this is a difficult task even for experienced human support agents.Thus, systems and methods are needed to generate replacement sentencesfor a given sentence to match a particular sentiment while substantiallyretaining the meaning of the sentence.

BRIEF SUMMARY

Certain embodiments provide a method for generating a replacementsentence with the same or similar meaning but a different sentiment thanan input sentence. The method generally includes receiving a request fora replacement sentence based on an input sentence and iterativelydetermining a next word of the replacement sentence word-by-word basedon the input sentence. Iteratively determining the next word generallyincludes evaluating a set of words of the input sentence using alanguage model configured to output candidate sentences based on aprobability of word appearance following the set of words of the inputsentence, the candidate sentences comprising the set of words appendedwith an ending word and evaluating the candidate sentences using asentiment model configured to output sentiment scores for the candidatessentences based on a sentiment of the candidate sentences. Iterativelydetermining the next word further includes calculating convexcombinations for the candidate sentences using the sentiment scores forthe candidate sentences and the probability of word appearance of thecandidate sentences and selecting, based on the convex combinations, anending word of one of the candidate sentences as the next word of thereplacement sentence. The method further includes transmitting, to achatbot server for display to a user, the replacement sentence inresponse to the request for the replacement sentence.

The following description and the related drawings set forth in detailcertain illustrative features of one or more embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The appended figures depict certain aspects of the one or moreembodiments and are therefore not to be considered limiting of the scopeof this disclosure.

FIG. 1 depicts a computing environment for generating candidatesentences based on an input sentence and a goal sentiment, according toone embodiment.

FIG. 2 depicts a computing environment for training a language model anda sentiment model, according to one embodiment.

FIG. 3 is a conceptual diagram of the operation of a bidirectional longshort term memory model, according to one embodiment.

FIG. 4 is a conceptual diagram showing a method of generating acandidate sentence, according to one embodiment.

FIG. 5 is a flow chart of example operations for generating a set ofcandidate sentences with the same or similar meaning but a differentsentiment than an input sentence.

FIG. 6 is a block diagram of example generation server, according to oneembodiment.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe drawings. It is contemplated that elements and features of oneembodiment may be beneficially incorporated in other embodiments withoutfurther recitation.

DETAILED DESCRIPTION

Aspects of the present disclosure provide apparatuses, methods,processing systems, and computer readable mediums for generating a setof candidate sentences with the same or similar meaning but a differentsentiment than an input sentence.

Real time customer assistance can be provided using tools including“chatbots” or other similar real time customer assistance systems, thatuse artificial intelligence (AI) and/or machine learning (ML) todetermine responses to, and interaction with, a customer. Such a chatbotcan provide an automated answering service that determines the bestanswers to questions and provide those answers to customer supportagents within existing communication applications. In this way, thecustomer support agent need not learn to use a new system, doindependent research, or otherwise depart from the customer supportagent’s typical workflow.

One limitation of chatbots and similar systems is the inability toprovide answers of a particular sentiment while retaining the meaning ofthe answers. While chatbot answers may be accurate for providingtechnical information, the sentiment expressed in an answer is alsoimportant. The sentiment used in customer support answers can impact howcustomers feel and experience a product, especially when such answersmay be provided for systems dealing with sensitive data. This problemaffects not just automated answers such as those from chatbots, but alsoanswers given by human agents, who may express inadvertent sentiment incustomer support answers.

Systems and methods to enable changing the sentiment of a sentence whileretaining meaning are described herein. In particular, the systems andmethods herein involve a machine learning model that, when given astring of characters from a sentence as input, outputs a set of wordsthat are most likely to follow the string. Because this set of words isthe most likely to follow the string, they likely represent a set ofalternative words that express the same meaning but may convey adifferent sentiment (e.g., the difference between the words “ask” and“demand.”). The machine learning model can thus generate candidatesentences for replacing an input sentence by building the candidatesentences word-by-word. Then, a different machine learning model trainedto analyze the in-context sentiment of sentences can be used to gradethe closeness of the candidate sentences to a goal sentiment. Thus, areplacement sentence can be determined, word-by-word, by maximizing theprobability of a next word and the closeness of the overall sentenceincluding the next word.

Consider the following simplified example: a user has requested helpfinding a particular user interface (UI) item in an application. Aninitial response by a chatbot used to respond to the request for helpmay be: “Button x is in the bottom right.” However, by using sentimentanalysis, this first generated response can be determined to haveneutral or slightly negative sentiment. If so, a user associated withthe chatbot (such as a developer) may request a replacement sentencethat retains the meaning of the initial response but that conveys a morepositive sentiment than the initial response.

By using the first words of the initial response (“Button x”) as input,a replacement sentence can be generated word-by-word using predictivemachine learning language model and a machine learning sentiment model.The replacement sentence may be: “Button x can be found in the lowerright.” This replacement sentence retains the meaning of the initialresponse (e.g., relative location of button x) but conveys a morepositive sentiment. As this sentence conveys a more positive tone thissentence may be able help the user more effectively compared to thefirst response. Additionally, this replacement sentence may improve theuser’s experience and attitude towards the provided customer helpoverall.

FIG. 1 depicts a computing environment 100 for generating candidatesentences based on an input sentence and a goal sentiment. Computingenvironment 100 includes computing device 120, generation server 130,generated response repository 134, chatbot server 170 and chatbot clientdevice 172, all connected via network 110.

Computing device 120 is a personal device such as a desktop computer,laptop computer or mobile device including at least a processor and amemory (not shown). In general, computing device 120 is used to initiatea request for a replacement sentence, based on a base sentence stored inmemory (input sentence 122) and a sentiment that the replacementsentence should match (requested sentiment 124). In this example,computing device 120 is used by a developer working on chatbot server170 to send replacement sentence requests to generation server 130. Inother examples, computing device 120 may be used or operated by acustomer support agent directly or generally by any user requesting agenerated sentence to maintain the meaning but alter the sentiment of abase sentence. In still other examples, chatbot server 170 may determineautonomously, when interacting with a customer, that a particular answerconveys an inappropriate sentiment (e.g.,. the sentiment of theparticular answer falls below a threshold for negativity express basedon a sentiment analysis). In such examples, chatbot server 170 canrequest a replacement sentence for the particular answer with a moreappropriate (e.g., more positive) sentiment, but the same meaning as theparticular answer. In general, a wide array of human users or softwareentities may request a sentence of a particular sentiment but retaininga meaning of an original sentence from generation server 130.

Generation server 130 is a computing device including at least aprocessor and a memory (not shown). Generation server 130 receivesreplacement sentence requests (such as from computing device 120) andgenerates new sentences in response to such requests. Generation server130 includes various software modules and routines such as languagemodel 140, sentiment model 150, score combiner 160 and sentence selector162, and stores various data elements, such as candidate sentences 142and sentiment scores 152. Although shown as a single entity, in otherexamples, the functions of generation server 130 may be performed acrossa plurality of additional computing devices, such as in a cloudcomputing system or other distributed computer system.

Language model 140 is a machine learning model executing on generationserver 130 to predict a next word in a sentence, based on one or moreprevious words. By so doing, language model 140 can generate a varietyof alternative to an input sentence, which express the same meaning asthe input sentence but a different sentiment. Language model 140 may beany of a variety of machine learning models suited for predictive tasks,including recurrent neural networks (RNN), long short term memory (LSTM)RNNs, bi-directional LSTM RNNs, time delay neural networks (TDNN),hidden Markov models (HMM), or others. In this example, language model140 is a bi-directional LSTM RNN. The functions of a bi-directional LSTMRNN are described in further detail below with respect to FIG. 3 .Further, in this example language model 140 may be trained on a separatecomputing device, such as training server 210 of FIG. 2 , below, but inother examples language model 140 can be trained an executed ongeneration server 130.

As an example of operation, when given the first word of a sentencelanguage model 140 predicts a second word, when given the first andsecond words language model 140 predicts a third word, and so on.Language model 140 can be so used to iteratively construct a sentenceword-by-word, by repeatedly generating a next word based on a previouslygenerated string or sentence fragment. Generally, language model 140outputs a set of most likely words rather than a single most likelyword, although in some examples language model 140 can produce a singleword instead. If outputting a set of most likely words, each word of theset of most likely words is associated with a probability of appearance,indicating a confidence or likelihood that the word should be the nextword in the sentence. Words with a high probability of appearanceindicate that the word likely retains the meaning of the word in thatposition of the input sentence.

Language model 140 produces candidate sentences 142 as output. Candidatesentences 142 are sentences, or partial sentences, intended as areplacement for input sentence 122, meaning they retain the meaning ofinput sentence 122 but convey a different sentiment than input sentence122. On a first pass, candidate sentences 142 may include as few as twowords each (one base word and one generated word) but as generationserver 130 continues to execute, language model 140 may continue toconstruct candidate sentences 142 to be arbitrarily long. In addition,using longer strings of base words may result in better retention of themeaning of the input sentence. For example, using the first ten words ofthe input sentence as base words may lead to a more maintained meaningthan using just the first word of the input sentence. In general,language model 140 may cease generation of candidate sentences 142 whenthe most recent word of candidate sentences 142 is an end of sentencetoken (such as a period, question mark or other punctuation) or whencandidate sentences 142 have exceeded a threshold value above the lengthof input sentence 122 (e.g., candidates sentences 142 are a certainpercentage longer than input sentence 122 or include a certain number ofwords more than input sentence 122).

Candidate sentences 142 are analyzed by sentiment model 150 to determinethe sentiment of candidate sentences 142. Sentiment model 150 is amachine learning model executing on generation server 130 to determinesentiment scores 152 for candidate sentences 142. Generally sentimentmodel 150 calculates an estimated “polarity” (e.g., positive ornegative) of an input sentence or sentence fragment. Sentiment model 150may use any of a variety of machine learning techniques, such as latentsemantic analysis, support vector machines (SVM), the bag of wordsmethod, deep learning techniques or other methods. Sentiment model 140may be trained on a separate computing device from generation server 130before execution.

Sentiment model 150 takes candidates sentences 142 as input and producessentiment scores 152 as output. In general, sentiment scores 152 areproduced for each of candidate sentences 142 graded relative torequested sentiment 124. Requested sentiment 124 may be scaled from 0(most negative) to 1 (most positive), with a score of 0.5 indicatingneutrality, although other scales (such as -1 to 1 or 0 to 100) arepossible. Sentiment scores 152 are a scoring of closeness to requestedsentiment 124. Thus, sentiment scores 152 may be a difference fromrequested sentiment 124 for each of candidate sentences 142 (e.g., theabsolute value of actual sentiment subtracted from requested sentiment),a relative closeness to requested sentiment 124 for each of candidatesentences 142 (e.g., a ranking of candidate sentences 142 based oncloseness) or other measures of closeness from one data point toanother.

Score combiner 160 is a software routine executing on generation server130 to generate a unified selection score for each of candidatesentences 142. In general, score combiner 160 may be a function tocalculate a convex combination for each of candidate sentences 142 usingthe probability of appearance for each candidate sentence (which servesas an approximation of closeness to meaning of the input sentence) andthe sentiment score for each candidate sentence. A convex combinationrefers to combination of points summing to 1. In this example, scorecombiner 160 may weigh the probability of selection for a given word andthe sentiment score for that word differently to afford more weight toone metric over the other (e.g., probability of appearance accountingfor 75% of the convex combination and sentiment score accounting for25%, or vice versa).

In some cases, alternatives to convex combinations may be used. Forexample, one alternative measure may be comparing the vector distance(e.g., using a distance function) between words of the base sentence andwords of candidate sentences 142, to establish semantic drift betweenthe base sentence and candidate sentences 142. Setting an upperthreshold for allowable semantic drift may ensure that the base sentenceand candidate sentences 142 are similar. This may allow for generatingsentences that change the fewest amount of words from the base sentence.

However, when using convex combinations, after score combiner 160produces the convex combinations for candidate sentences 142, sentenceselector 162 identifies a sentence (or set of sentences) to advance tothe next iterative word generation. For example, if language model 140produces five candidate sentences (the same base sentence ending in fivenewly generated words) sentence selector 162 may identify a singlesentence to advance. In general score combiner 160 may use a variety ofdifferent searching or sorting methods to identify the highest scoringsentences to advance, such as beam search, B* search, best-first search,depth-first search, Dijkstra’s algorithm or other searching algorithmsor methods.

Generation server 130 is connected to generated response repository 134.Generated response repository 134 is a database or other data storagedevice used to store responses previously generated by generation server130. Generated response repository 134 is shown as a separate entityconnected to generation server 130, but in other examples, generatedresponse repository 134 may be a storage device within generation server130, or may be a storage device available to generation server 130 vianetwork 110. In some examples, generated response repository 134 may beaccessed by generation server 130 to retrieve previously generatedresponses that match an incoming request to generation server 130. Forexample, if generation server 130 receives a request duplicative of aprevious request, generation server 130 may retrieve a responsepreviously generated for the previous request, rather than regenerate aresponse for the duplicate request.

The overall operation of the components of generation server 130 is asfollows: after receiving a request for a replacement sentence based oninput sentence 122, language model 140 produces a plurality ofcandidates sentences 142 based on words of the sentence being replacedand ending in a newly generated word. Sentiment model 150 then producessentiment scores 152 based on the closeness of the sentiment ofcandidate sentences 142 to requested sentiment 124. Then, score combiner160 produces a set of convex combinations based on the probability ofword appearance for each of candidate sentences 142 as well as sentimentscores 152. Then, sentence selector 162 identifies a candidate sentenceto advance to the next iterative round, the selected candidate sentenceacting as the base sentence for the next round of selection by languagemodel 140. This process repeats until the candidate sentence selected bysentence selector 162 ends in an end-of-sentence token, or until thecandidate sentence selected by sentence selector 162 exceeds a lengththreshold based on the length of input sentence 122.

When generation server 130 has finished generation of a replacementsentence, the replacement sentence may be transmitted to computingdevice 120 in response to the initial request. Computing device 120 (ora user of computing device 120) may then use the replacement sentence ina variety of contexts. In this example, the user of computing device 120provides the replacement sentence to chatbot server 170. Chatbot server170 is a computing device executing a chatbot service or program. Duringexecution, chatbot server 170 may use the replacement sentence in orderto respond to a query from an end user or a customer operating chatbotclient device 172. In other examples, the user of computing device 120may use the replacement sentence when supporting the end user or thecustomer directly.

FIG. 2 depicts a computing environment 200 for training a language modeland a sentiment model, such as language model 140 and sentiment model150, respectively. Computing environment 200 includes generation server130, training server 210 and conversation repository 240.

Training server 210 is a computing device including at least a processorand memory (not shown). Training server 210 can be used to trainlanguage model 140, sentiment model 150 or other machine learningmodels. In other examples, language model 140 and sentiment model 150may be trained on generation server 130 or another computing device.Additionally, the functions of training server 210 may be performed bymultiple additional computing devices such as in a cloud computingsystem or other distributed computer system. Training server 210includes two software modules, language module 220 for use in traininglanguage model 140, and sentiment module 230 for use in trainingsentiment model 150.

Training server 210 is connected to conversation repository 240.Conversation repository 240 is a database or other data storage deviceused to store data, such as text transcriptions, related to priorcustomer help conversations between human support agents and customersand end-users. In this example, the data of conversation repository 240is used as a source of training data for training language model 140.Using historical conversations as training data enables language model140 when fully trained to be adapted to a particular domain associatedwith the customer help conversations. In other examples, however,different sources of training data may be used, such as other databasesor repositories of human language sentences. Conversation repository 240is shown as a separate entity connected to training server 210, but inother examples, generated conversation repository 240 may be a storagedevice within training server 210, or may be a storage device availableto training server 210 via a network connection.

Language module 220 is a software module executing on training server210 including feature extractor 222 and language model 140 assubroutines, and word vectors 224 as stored data. Feature extractor 222is a software routine that performs various feature extractionoperations on the data of conversation repository 240 to generate wordvectors 224. Feature extraction is a process to isolate meaningfulaspects of raw data, and to remove irrelevant or meaningless aspects ofraw data, such as punctuation or other unintelligible information.Feature extraction also includes formatting the meaningful aspects ofraw data into a computer-readable format, such as, for example,numerical or vector representations of words and other document content.

Feature extraction may include various preprocessing operations ondocument content and text, such as removing stop words, performingtokenization, N-gram processing, performing stemming, removingnon-English words, removing punctuation, replacing hypertext markuplanguage (HTML) syntax with English grammar or removing personallyidentifiable information (PII) from the text.

One preprocessing operation is stemming. Stemming is the process ofreducing inflected (or sometimes derived) words to their word stem,base, or root form. Stemming algorithms such as Porter’s Algorithm andothers may be applied during stemming. Lemmatization is a more complexapproach to determining a stem of a word that involves first determiningthe part of speech of a word, and applying different normalization rulesfor each part of speech. In lemmatization, the part of speech is firstdetected prior to attempting to find the root since for some languages,the stemming rules change depending on a word’s part of speech.

After performing preprocessing, vectors may be generated from thepreprocessed content. For example, the vectors may be multi-dimensional,with values for each dimension randomly initialized. These vectors maycorrespond to a particular word of the preprocessed content. In general,vectors may be mapped to particular words using an embedding matrix, amatrix of all words in the system and their corresponding values (alsocalled embeddings). By randomly initializing the dimensional values ofvectors, the values can be refined during the training process, whichcan adapt the vectors to the particular vocabulary or domain associatedwith the original document content. In other cases, established vectorsystems, such as Global Vector for Word Representation (GloVe) orWord2vec, may be used in place of randomly initialized vector values.Both GloVe and Word2vec utilize three hundred-dimensional vectorrepresentations of words.

Word vectors 224 are set of vectors representing words from conversationrepository 240. Word vectors 224 are used as input to train languagemodel 140. Word vectors 224 may be stored as a sequence of vectorscorresponding to sentences form conversation repository 240. Maintainingthe sequence of sentences also maintains the context and meaning of thesentences, allowing language model 140 to take the context of individualwords into account. The values of word vectors 224 may be randomlyinitialized, as doing so allows word vectors 224 to be refined duringthe process of training language model 140. When doing so, the values ofword vectors 224 can be adapted to the particular domain covered bylanguage model 140 (e.g., if language model 140 is used to generatereplacement sentences for a financial services application, word vectors224 may be specialized to reflect vocabulary of the domain of financialservices). In other examples, word vectors 224 may be obtained from anexternal service with predefined values.

In general, training a machine learning model involves providing testinput (such as a vector representing a word) to the machine learningmodel, with the machine learning model producing output as a result.This output is compared to a label for the test input, and the machinelearning model is adjusted to produce output closer to the label. Thisprocess is repeated iteratively until the machine learning modelproduces sufficiently close output (according to a threshold or theopinion of an operator).

In this example, one or more vectors of word vectors 224 are provided tolanguage model 140 as input. Different length sequences may be used toretain more context of the sequence provided. In some examples, 15 wordsequences may be provided. Then, language model 140 produces a predictednext word as output. Following the example above, if a 15 word sequenceis provided, language model 140 attempts to produce a 16^(th) word. Theactual next word following the provided sequence is used as the labelfor the input. Thus, if the output matches the label, language model 140has correctly predicted the next word of the sentence. By modifyingparameters of language model 140 when language model 140 does notcorrectly predict the next word, language model 140 can be iterativelyimproved until language model 140 meets a threshold for accuratepredictions. One potential threshold may be 95% (e.g., language model140 correctly predicts a next word 95% of the time), although many otherthresholds may be used at the discretion of the operator of trainingserver 210.

Sentiment module 230 is software module executing on training server 210including corpus 232 as stored data and sentiment model 150 as asubroutine. Corpus 232 is a collection of data used in trainingsentiment model 150. In this example corpus 232 is a large collection ofsentences each labeled with a measure of sentiment. The label used maydepend on the method of calculating sentiment employed. For example, ifsentiment polarity is used, the labels for each sentence may be scaledfrom -1 to 1 or other scales as discussed above. In general, sentencesof corpus 232 are provided to sentiment model 150 as input, andsentiment model 150 produces a predicted label for the sentences asoutput. Parameters of sentiment model 150 are iteratively modified basedon the closeness of the output to the label until sentiment model 150reaches a threshold for correct prediction. In this example sentimentmodel 150 is trained on training server 210, but in other examplespre-trained sentiment models may be obtained for use, or externalsentiment services can be used.

In this example, training server 210 trains both language model 140 andsentiment model 150, and when fully trained, transfers language model140 and sentiment model 150 to generation server 130 for execution, asdiscussed in further detail above. In other examples, different devices(including generation server 130) may be used to train language model140 and sentiment model 150 separately, or language model 140 andsentiment model 150 may be trained by a cloud computing system or otherdistributed computing system.

FIG. 3 is a conceptual diagram 300 of the operation of a bidirectionallong short term memory model (Bi-LSTM). In some examples, a languagemodel used for generating replacement sentences (such as language model140 of FIG. 1 ), may be a Bi-LSTM. Diagram 300 includes four vectorrepresentations of words, 310, 320, 330 and 340, which in this examplemay correspond to words of a four word sentence. Diagram 300 alsoincludes LSTM cells 1-8 and four context analyses 315, 325, 335 and 345,each representing a predicted next word for each of the vectorrepresentations. The four context analyses have two sections, a forwardsprediction (shown as the bottom of each context analysis andrepresenting a predicted word given the forwards context of a sentence)and a backwards analysis (shown as the top of each context analysis andrepresenting a predicted word given the backwards context of asentence).

A Bi-LSTM consists of a number of LSTM cells, which each take one of thevector representations as input. Each LSTM cell also pushes forward itsoutput to the next LSTM in sequence. Because the Bi-LSTM isbidirectional, it includes two series of LSTM cells, shown as 350 and355. The Bi-LSTM model in diagram 300 includes eight individual LSTMcells, four as the forward direction (350) and four as the backwarddirection (355). The exact number of LSTM cells in a Bi-LSTM model isarbitrary, however.

In operation, LSTM 1 of 350 takes vector 310 as input. LSTM 1 thenoutputs the context analysis of vector 310 to both 315 and to LSTM 2.LSTM 2 takes the context analysis of vector 310 as well as vector 320 asinput. Thus, the context analysis of vector 320 takes into account thecontext analysis of vector 310. LSTM 2 then outputs the context analysisof vector 320 to both LSTM 3 and 325.

This process continues for LSTM 3 and LSTM 4, so that the contextanalysis of vector 330 (shown as the bottom half of 335) includes thecontext analysis of vectors 310 and 320, and the context analysis ofvector 340 includes the context analysis of vectors 310, 320 and 330.For example, if vectors 310, 320, 330 and 340 represent the words “howare you doing,” the forwards context analysis at 315 is for “how,” theforwards context analysis at 325 is for “how are,” the forwards contextanalysis at 335 is for “how are you,” and the forwards context analysisat 345 is for “how are you doing.” Based on the forwards contextanalysis at 335, a predicted next word may be “today,” while the morelimited forwards context analysis at 315, a predicted next word may be“is.” In general, the more context provided, the better qualityprediction may be produced.

A similar process is performed by LSTM 5-8 for the backwards direction,resulting in both a forwards and backwards context analysis for each ofthe four vectors. Performing a backwards analysis may improve thecontext analysis for some sentence structures, for example when the lastword in a sentence is a significant word contextually.

FIG. 4 is a conceptual diagram 400 showing a method of generating acandidate sentence. Diagram 400 illustrates the generation of a sentenceusing a tree structure, specifically with tree levels 415, 420, 430,440, 450 and 460. Diagram 400 also includes sentence information 410,and candidate sentences 425, 435, 445, 455 and 465. In this example theprobabilities of appearance and sentiment shown are merely illustrativeand do not reflect actual calculations of probability or sentiment forthe given contexts.

Sentence information 410 summarizes the details of a sentence generationrequest as discussed above. In particular, sentence information 410includes the input sentence “What is your problem?), the currentsentiment of that sentence (-.5, moderately negative) and the goalsentiment (0.5, moderately positive).

Based on the input sentence, a generation server, such as generationserver 130 of FIG. 1 , attempts to generate a replacement sentence. Inthis example, the first word of the input sentence (“what”) is used asbase sentence as shown at 415, although in other examples longersequences or sentence fragments may be used as a base sentence. Here,the generation server uses a language model as described above togenerate candidate sentences 425 based on the base sentence. As shown attree level 420, one additional word has been added to the base sentenceas shown to generate candidate sentences 425. Each additional word isassociated with the probability of word appearance. In particular, theword “are” has a probability of word appearance of 25%, the word “can”has a probability of word appearance of 30% and the word “is” has aprobability of word appearance of 20%.

Each word shown at 420 corresponds with a sentence of candidatesentences 425. Candidate sentences 425 are “What are,” “What can” and“What is.” After candidate sentences 425 have been generated, thegeneration server uses a sentiment model to analyze the sentiment ofeach candidate sentence. The sentiment scores for the candidatesentences is shown to the right of each candidate sentence. Inparticular, the sentence “What are” has a sentiment score of 0.1(slightly positive), the sentence “What can” has a sentiment score of0.2 and the sentence “What is” has a sentiment score of 0 (neutral).

After calculating the sentiment scores, the generation server determinesa candidate sentence of candidate sentences to use as the base sentencefor a next execution of the language model. In this case, the generationserver selects “What can,” which is associated with the highestprobability of appearance (30%), and the sentiment closest to the goalsentiment (0.2 with the goal of 0.5). The word “can” being selected isillustrated by the outline around the word “can” at 420.

On a second pass, the language model uses “What can” as the basesentence, producing three words at 430. The words produced are “we,”with a probability of appearance of 35%, “I,” with a probability ofappearance of 20% and “you,” with a probability of appearance of 10%.These words correspond with candidate sentences 435 appending eachproduced word to the base sentence, and each have a sentiment score of0.2. Because each candidate sentence has the same sentiment score,generation server selects the candidate sentence associated with thehighest probability of appearance, “What can we.”

On a third pass, the language model uses “What can we” as the basesentence, producing three words at 440. The words produced are “help,”with a probability of appearance of 25%, “assist,” with a probability ofappearance of 25% and “do,” with a probability of appearance of 10%.Unlike candidate sentences 435, each of candidate sentences 445 has adifferent sentiment score. Because the sentences “What can we help” and“What can we assist” are associated with the same probability ofappearance, the generation server selects the candidate sentence withthe highest sentiment score, “What can we assist.”

On a fourth pass, the language model uses “What can we assist” as thebase sentence, producing three words at 450. The words produced are“for,” with a probability of appearance of 5%, “with,” with aprobability of appearance of 20% and “you,” with a probability ofappearance of 40%. Each of candidate sentences 455 has the samesentiment score, 0.3. Thus, the generation server selects the candidatesentence associated with the highest probability of appearance, “Whatcan we assist you.”

Finally, on a fifth pass, the language model uses “What can we assistyou” as the base sentence, producing three words at 460. The wordsproduced are “with,” with a probability of appearance of 75%, “on,” witha probability of appearance of 5% and “for,” with a probability ofappearance of 5%. As with candidate sentences 455, each of candidatesentences 465 has the same sentiment score, 0.3. Thus, the generationserver once again selects the candidate sentence associated with thehighest probability of appearance, “What can we assist you with.”

As can be shown with the resulting replacement sentence “What can weassist you with,” by generating the replacement sentence word-by-word, asentence with a retained meaning to the input sentence “What is yourproblem?” is obtained. Further, the replacement sentence conveys a morepositive sentiment than does the input sentence, and so may be moreappropriate for some contexts, such as use in a customer help orcustomer support service.

In this example, the generation server produces 3 candidate sentences ateach pass, however, in other examples the generation server may producemore or fewer candidate sentences. Additionally, rather than selecting asingle candidate sentence to advance, the generation server may advancemultiple candidate sentences at each pass.

FIG. 5 is a flow chart of example operations 500 for generating areplacement sentence with the same or similar meaning but a differentsentiment than an input sentence. Operations 500 may be performed by ageneration server, such as generation server 130 of FIG. 1 .

Operations 500 begin at 510, where the generation server receives arequest for a replacement sentence based on an input sentence. Asdiscussed above, the request may come from a variety of sources,including a chatbot application or chatbot server, a computing deviceassociated with a developer working on a chatbot system, or anycomputing device operated by a user seeking a replacement sentencegenerally. The generation server responds to the request by iterativelydetermining words of the replacement sentence word-by-word based on theinput sentence.

At 520, the generation server evaluates a set of words of the inputsentence using a language model configured to output candidate sentencesbased on a probability of word appearance following the set of words ofthe input sentence. The candidate sentences comprise the set of words ofthe input sentence appended with an ending word, the ending word beinggenerated by the language model. In general, by outputting words basedon the probability of word appearance, the language model outputs wordsthat retain the original meaning of the input sentence, although thewords may express a different sentiment.

At 530, the generation server evaluates the candidate sentences using asentiment model configured to output sentiment scores for the candidatessentences based on a sentiment of the candidate sentences. As discussedabove, the sentiment scores may reflect the polarity of sentiment of thecandidate sentences. Further, the sentiment scores may be based on thedifference between the actual sentiment of the candidate sentences and agoal sentiment included in the request for the replacement sentence.

At 540 the generation server calculates convex combinations for thecandidate sentences using the sentiment scores for the candidatesentences and the probability of word appearance of the candidatesentences. As discussed above, convex combinations are a calculation ofone or more values summing to 1, allowing the relative weight accordedto the one or more values to be adjusted. In other examples, othertechniques for combining the probability of word appearance and thesentence scores may be used.

At 550 the generation server selects, based on the convex combinations,an ending word of one of the candidate sentences as the next word of thereplacement sentence. In some embodiments, selecting the next word ofthe replacement sentence comprises identifying the ending wordsassociated with the highest value convex combinations using a beamsearch algorithm. In other examples other search methods or searchalgorithms may be used, as discussed above.

At 560 the generation server determines if the end of the replacementsentence has been reached. For example, in some cases the generationserver ends iteratively determining the next word of the replacementsentence when the next word of the replacement sentence is an end ofsentence token. In other cases the generation server ends iterativelydetermining the next word of the replacement sentence when thereplacement sentence has length exceeding a length threshold based onlength of the input sentence. If the end of the replacement sentence hasnot been reached, operations 500 return to 520, where an additional nextword is determined for the replacement sentence. If the end of thereplacement sentence has been reached, operations 500 proceed to 570.

At 570, the generation server transmits the replacement sentence inresponse to the request for the replacement sentence. For example, thegeneration server may transmit the replacement sentence to a chatbotserver for display to a user. In other examples the generation servermay transmit the replacement sentence to a variety of other computingdevices.

Some examples of operations 500 further include randomly initializingvector representations of words for use in training the language model,and during training of the language model, refining the vectorrepresentations of words based on results of the training.

Other examples of operations 500 further include receiving a secondrequest for a second replacement sentence with a requested sentiment andan original meaning; and responding to the second request using apreviously generated sentence stored in a response database, wherein thepreviously generated sentence has the requested sentiment and theoriginal meaning.

In some cases, the language model is a bi-directional long short-termmemory (Bi-LSTM) model trained with a set of historical conversationsbetween human support agents and users. In such cases operations 500 mayfurther include training the language model by providing the languagemodel with a series of a fifteen word sequences taken from the set ofhistorical conversations, wherein a label for each fifteen word sequenceis a sixteenth word following the fifteen word sequence.

In some examples of operations 500, the sentiment model calculates thesentiment scores based on a goal sentiment provided with the inputsentence, wherein the goal sentiment is different from a sentiment ofthe input sentence. In such examples, the probability of word appearancefor a candidate sentence indicates a probability that an ending word ofthe candidate sentence would maintain meaning of the input sentence.

FIG. 6 is a block diagram of example generation server 600. As shown,the generation server 600 includes, without limitation, a centralprocessing unit (CPU) 602, one or more input/output (I/O) deviceinterfaces 604, which may allow for the connection of various I/Odevices 614 (e.g., keyboards, displays, mouse devices, pen input, etc.)to generation server 600, network interface 606, memory 608, storage610, and an interconnect 612.

The CPU 602 may retrieve and execute programming instructions stored inthe memory 608. Similarly, the CPU 602 may retrieve and storeapplication data residing in the memory 608. The interconnect 612transmits programming instructions and application data, among the CPU602, I/O device interface 604, network interface 606, memory 608, andstorage 610. The CPU 602 is included to be representative of a singleCPU, multiple CPUs, a single CPU having multiple processing cores, andthe like. The I/O device interface 604 may provide an interface forcapturing data from one or more input devices integrated into orconnected to the generation server 600, such as keyboards, mice,touchscreens, and so on. The memory 608 may represent a random accessmemory (RAM), while the storage 610 may be a solid state drive, forexample. Although shown as a single unit, the storage 610 may be acombination of fixed and/or removable storage devices, such as fixeddrives, removable memory cards, network attached storage (NAS), orcloud-based storage.

As shown, the memory 608 includes language model 621, sentiment model622, score combiner 623 and sentence selector 624, which are softwareroutines executed based on instructions stored in the storage 610. Suchinstructions may be executed by the CPU 602.

As shown, the storage 610 includes input sentence 631, candidatesentences 632, requested sentiment 633 and sentiment scores 634.Language model 621 evaluates input sentence 631 to produce candidatessentences 632 based on a probability of word appearance. Then, sentimentmodel evaluates the sentiment scores of candidate sentences 632 relativeto requested sentiment 633 to produce sentiment scores 634. Then, scorecombiner 623 combines sentiment scores 634 and the probability of wordappearance for candidate sentences 632 to produce combined scores. Then,based on the combined scores, sentence selector 624 chooses a new wordfor use as a next word in a replacement sentence. When the replacementsentence is complete, it may be transmitted from generation server 600to a different computing device, such as via network interface 606.

The preceding description is provided to enable any person skilled inthe art to practice the various embodiments described herein. Theexamples discussed herein are not limiting of the scope, applicability,or embodiments set forth in the claims. Various modifications to theseembodiments will be readily apparent to those skilled in the art, andthe generic principles defined herein may be applied to otherembodiments. For example, changes may be made in the function andarrangement of elements discussed without departing from the scope ofthe disclosure. Various examples may omit, substitute, or add variousprocedures or components as appropriate. For instance, the methodsdescribed may be performed in an order different from that described,and various steps may be added, omitted, or combined. Also, featuresdescribed with respect to some examples may be combined in some otherexamples. For example, an apparatus may be implemented or a method maybe practiced using any number of the aspects set forth herein. Inaddition, the scope of the disclosure is intended to cover such anapparatus or method that is practiced using other structure,functionality, or structure and functionality in addition to, or otherthan, the various aspects of the disclosure set forth herein. It shouldbe understood that any aspect of the disclosure disclosed herein may beembodied by one or more elements of a claim.

As used herein, the term “determining” encompasses a wide variety ofactions. For example, “determining” may include calculating, computing,processing, deriving, investigating, looking up (e.g., looking up in atable, a database or another data structure), ascertaining and the like.Also, “determining” may include receiving (e.g., receiving information),accessing (e.g., accessing data in a memory) and the like. Also,“determining” may include resolving, selecting, choosing, establishingand the like.

The methods disclosed herein comprise one or more steps or actions forachieving the methods. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims. Further, thevarious operations of methods described above may be performed by anysuitable means capable of performing the corresponding functions. Themeans may include various hardware and/or software component(s) and/ormodule(s), including, but not limited to a circuit, an applicationspecific integrated circuit (ASIC), or processor. Generally, where thereare operations illustrated in figures, those operations may havecorresponding counterpart means-plus-function components with similarnumbering.

The various illustrative logical blocks, modules and circuits describedin connection with the present disclosure may be implemented orperformed with a general purpose processor, a digital signal processor(DSP), an application specific integrated circuit (ASIC), a fieldprogrammable gate array (FPGA) or other programmable logic device (PLD),discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein.A general-purpose processor may be a microprocessor, but in thealternative, the processor may be any commercially available processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration.

A processing system may be implemented with a bus architecture. The busmay include any number of interconnecting buses and bridges depending onthe specific application of the processing system and the overall designconstraints. The bus may link together various circuits including aprocessor, machine-readable media, and input/output devices, amongothers. A user interface (e.g., keypad, display, mouse, joystick, etc.)may also be connected to the bus. The bus may also link various othercircuits such as timing sources, peripherals, voltage regulators, powermanagement circuits, and other circuit elements that are well known inthe art, and therefore, will not be described any further. The processormay be implemented with one or more general-purpose and/orspecial-purpose processors. Examples include microprocessors,microcontrollers, DSP processors, and other circuitry that can executesoftware. Those skilled in the art will recognize how best to implementthe described functionality for the processing system depending on theparticular application and the overall design constraints imposed on theoverall system.

If implemented in software, the functions may be stored or transmittedover as one or more instructions or code on a computer-readable medium.Software shall be construed broadly to mean instructions, data, or anycombination thereof, whether referred to as software, firmware,middleware, microcode, hardware description language, or otherwise.Computer-readable media include both computer storage media andcommunication media, such as any medium that facilitates transfer of acomputer program from one place to another. The processor may beresponsible for managing the bus and general processing, including theexecution of software modules stored on the computer-readable storagemedia. A computer-readable storage medium may be coupled to a processorsuch that the processor can read information from, and write informationto, the storage medium. In the alternative, the storage medium may beintegral to the processor. By way of example, the computer-readablemedia may include a transmission line, a carrier wave modulated by data,and/or a computer readable storage medium with instructions storedthereon separate from the wireless node, all of which may be accessed bythe processor through the bus interface. Alternatively, or in addition,the computer-readable media, or any portion thereof, may be integratedinto the processor, such as the case may be with cache and/or generalregister files. Examples of machine-readable storage media may include,by way of example, RAM (Random Access Memory), flash memory, ROM (ReadOnly Memory), PROM (Programmable Read-Only Memory), EPROM (ErasableProgrammable Read-Only Memory), EEPROM (Electrically ErasableProgrammable Read-Only Memory), registers, magnetic disks, opticaldisks, hard drives, or any other suitable storage medium, or anycombination thereof. The machine-readable media may be embodied in acomputer-program product.

A software module may comprise a single instruction, or manyinstructions, and may be distributed over several different codesegments, among different programs, and across multiple storage media.The computer-readable media may comprise a number of software modules.The software modules include instructions that, when executed by anapparatus such as a processor, cause the processing system to performvarious functions. The software modules may include a transmissionmodule and a receiving module. Each software module may reside in asingle storage device or be distributed across multiple storage devices.By way of example, a software module may be loaded into RAM from a harddrive when a triggering event occurs. During execution of the softwaremodule, the processor may load some of the instructions into cache toincrease access speed. One or more cache lines may then be loaded into ageneral register file for execution by the processor. When referring tothe functionality of a software module, it will be understood that suchfunctionality is implemented by the processor when executinginstructions from that software module.

The following claims are not intended to be limited to the embodimentsshown herein, but are to be accorded the full scope consistent with thelanguage of the claims. Within a claim, reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. No claim element is tobe construed under the provisions of 35 U.S.C. §112(f) unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” All structural and functional equivalents to the elements of thevarious aspects described throughout this disclosure that are known orlater come to be known to those of ordinary skill in the art areexpressly incorporated herein by reference and are intended to beencompassed by the claims. Moreover, nothing disclosed herein isintended to be dedicated to the public regardless of whether suchdisclosure is explicitly recited in the claims.

What is claimed is:
 1. A method for transforming sentiment in text,comprising: receiving a request for a replacement sentence based on aninput sentence with a given sentiment, wherein the replacement sentencehas a different sentiment than the given sentiment; receiving, from alanguage model based on the input sentence, a set of candidatesentences; determining a meaning of each respective candidate sentenceof the set of candidate sentences based at least on one or more words ofthe respective candidate sentence; receiving, from a sentiment modelbased on the set of candidate sentences, a sentiment of each candidatesentence of the set of candidate sentences; choosing a particularcandidate sentence of the set of candidate sentences based on thedetermined meaning of the particular candidate sentence and the receivedsentiment of the particular candidate sentence; and providing theparticular candidate sentence in response to the request for thereplacement sentence.
 2. The method of claim 1, further comprising:randomly initializing vector representations of words for use intraining the language model; and during training of the language model,refining the vector representations of words based on results of thetraining.
 3. The method of claim 1, wherein choosing the particularcandidate sentence of the set of candidate sentences based on thedetermined meaning of the particular candidate sentence and the receivedsentiment of the particular candidate sentence comprises: calculating aconvex combination for each candidate sentence of the set of candidatesentences based on the sentiment of each candidate sentence of the setof candidate sentences and the meaning of each respective candidatesentence of the set of candidate sentences; and choosing the particularcandidate sentence based on a highest convex combination for theparticular candidate sentence.
 4. The method of claim 1, furthercomprising: receiving an additional request for an additionalreplacement sentence with the different sentiment; and providing theparticular candidate sentence in response to receiving the additionalrequest.
 5. The method of claim 1, wherein the language model is abi-directional long short-term memory (Bi-LSTM) model trained with a setof historical conversations between human support agents and users. 6.The method of claim 5, further comprising training the language model byproviding the language model with a series of fifteen word sequencestaken from the set of historical conversations, wherein a label for eachfifteen word sequence is a sixteenth word following the fifteen wordsequence.
 7. The method of claim 1, wherein the sentiment modelcalculates the sentiment of each candidate sentence of the set ofcandidate sentences based on a requested sentiment, wherein therequested sentiment is: provided to the sentiment model with the inputsentence, and is different from the given sentiment.
 8. The method ofclaim 1, wherein: providing the particular candidate sentence inresponse to the request for the replacement sentence comprises providingthe particular candidate sentence to the language model; and the methodfurther comprises: receiving, based on the particular candidate sentenceand the input sentence, an additional set of candidate sentences,wherein: each candidate sentence of the additional set of candidatesentences comprises the particular candidate sentence and an endingword, and the ending word of each candidate sentence of the additionalset of candidate sentences is different than each other candidatesentence in the additional set of candidate sentences; choosing anadditional candidate sentence of the additional set of candidatesentences based on a meaning of the additional candidate sentence and asentiment of the additional candidate sentence; and providing theadditional candidate sentence to a user associated with the differentsentiment.
 9. The method of claim 1, wherein providing the particularcandidate sentence in response to the request for the replacementsentence comprises providing the particular candidate sentence to a userassociated with the request.
 10. The method of claim 1, wherein: thedetermined meaning of the particular candidate sentence is compared to ameaning of the input sentence to determine if the determined meaning ofthe particular candidate sentence meets a first threshold; and thereceived sentiment of the particular candidate sentence is compared to arequested sentiment to determine if the received sentiment of theparticular candidate sentence meets a second threshold; and providingthe particular candidate sentence in response to the request for thereplacement sentence the determined meaning of the particular candidatesentence and the received sentiment of the particular candidate sentencecomprises providing the particular candidate sentence based on whetherthe determined meaning of the particular candidate sentence meets thefirst threshold and whether the received sentiment of the particularcandidate sentence meets the second threshold.
 11. The method of claim1, further comprising determining a maximum length for each candidatesentence of the set of candidate sentences based on the input sentence,wherein no candidate sentence of the set of candidate sentences exceedsthe maximum length.
 12. A system, comprising: one or more processors;and a memory storing instructions that, when executed by the one or moreprocessors, cause the system to: receive a request for a replacementsentence based on an input sentence with a given sentiment, wherein thereplacement sentence has a different sentiment than the given sentiment;receive, from a language model based on the input sentence, a set ofcandidate sentences; determine a meaning of each respective candidatesentence of the set of candidate sentences based at least on one or morewords of the respective candidate sentence; receive, from a sentimentmodel based on the set of candidate sentences, a sentiment of eachcandidate sentence of the set of candidate sentences; choose aparticular candidate sentence of the set of candidate sentences based onthe determined meaning of the particular candidate sentence and thereceived sentiment of the particular candidate sentence; and provide theparticular candidate sentence in response to the request for thereplacement sentence.
 13. The system of claim 12, wherein theinstructions, when executed by the one or more processors, further causethe system to: randomly initialize vector representations of words foruse in training the language model; and during training of the languagemodel, refine the vector representations of words based on results ofthe training.
 14. The system of claim 12, wherein choosing theparticular candidate sentence of the set of candidate sentences based onthe determined meaning of the particular candidate sentence and thereceived sentiment of the particular candidate sentence comprises:calculating a convex combination for each candidate sentence of the setof candidate sentences based on the sentiment of each candidate sentenceof the set of candidate sentences and the meaning of each respectivecandidate sentence of the set of candidate sentences; and choosing theparticular candidate sentence based on a highest convex combination forthe particular candidate sentence.
 15. The system of claim 12, whereinthe instructions, when executed by the one or more processors, furthercause the system to: receive an additional request for an additionalreplacement sentence with the different sentiment; and provide theparticular candidate sentence in response to receiving the additionalrequest.
 16. The system of claim 12, wherein the language model is abi-directional long short-term memory (Bi-LSTM) model trained with a setof historical conversations between human support agents and users. 17.The system of claim 16, wherein the instructions, when executed by theone or more processors, further cause the system to train the languagemodel by providing the language model with a series of fifteen wordsequences taken from the set of historical conversations, wherein alabel for each fifteen word sequence is a sixteenth word following thefifteen word sequence.
 18. The system of claim 12, wherein the sentimentmodel calculates the sentiment of each candidate sentence of the set ofcandidate sentences based on a requested sentiment, wherein therequested sentiment is: provided to the sentiment model with the inputsentence, and is different from the given sentiment.
 19. The system ofclaim 12, wherein: providing the particular candidate sentence inresponse to the request for the replacement sentence comprises providingthe particular candidate sentence to the language model; and theinstructions, when executed by the one or more processors, further causethe system to: receive, based on the particular candidate sentence andthe input sentence, an additional set of candidate sentences, wherein:each candidate sentence of the additional set of candidate sentencescomprises the particular candidate sentence and an ending word, and theending word of each candidate sentence of the additional set ofcandidate sentences is different than each other candidate sentence inthe additional set of candidate sentences; choose an additionalcandidate sentence of the additional set of candidate sentences based ona meaning of the additional candidate sentence and a sentiment of theadditional candidate sentence; and provide the additional candidatesentence to a user associated with the different sentiment.
 20. A methodfor transforming sentiment in text, comprising: receiving a request fora replacement sentence based on an input sentence with a givensentiment, wherein the replacement sentence has a different sentimentthan the given sentiment; receiving, from a language model based on theinput sentence, a set of candidate sentences; receiving, from asentiment model based on the set of candidate sentences, a sentiment ofeach candidate sentence of the set of candidate sentences; determine asemantic drift between the input sentence and each respective candidatesentence of the set of candidate sentences based on comparing a vectordistance between words of the input sentence and words of eachrespective candidate sentence; choosing a particular candidate sentenceof the set of candidate sentences based on the determined semantic driftof the particular candidate sentence and a received sentiment of theparticular candidate sentence; comparing the determined semantic driftof the particular candidate sentence to a threshold; and providing theparticular candidate sentence in response to the request for thereplacement sentence based on comparing the determined semantic drift ofthe particular candidate sentence to the threshold.